Your Model Is Predictive - but Is It Useful? Theoretical and Empirical Considerations of a New Paradigm for Adaptive Tutoring Evaluation
نویسندگان
چکیده
Classification evaluation metrics are often used to evaluate adaptive tutoring systems— programs that teach and adapt to humans. Unfortunately, it is not clear how intuitive these metrics are for practitioners with little machine learning background. Moreover, our experiments suggest that existing convention for evaluating tutoring systems may lead to suboptimal decisions. We propose the Learner Effort-Outcomes Paradigm (Leopard), a new framework to evaluate adaptive tutoring. We introduce Teal and White, novel automatic metrics that apply Leopard and quantify the amount of effort required to achieve a learning outcome. Our experiments suggest that our metrics are a better alternative for evaluating adaptive tutoring.
منابع مشابه
Adaptive Simplified Model Predictive Control with Tuning Considerations
Model predictive controller is widely used in industrial plants. Uncertainty is one of the critical issues in real systems. In this paper, the direct adaptive Simplified Model Predictive Control (SMPC) is proposed for unknown or time varying plants with uncertainties. By estimating the plant step response in each sample, the controller is designed and the controller coefficients are directly ca...
متن کاملAdaptive Tuning of Model Predictive Control Parameters based on Analytical Results
In dealing with model predictive controllers (MPC), controller tuning is a key design step. Various tuning methods are proposed in the literature which can be categorized as heuristic, numerical and analytical methods. Among the available tuning methods, analytical approaches are more interesting and useful. This paper is based on a proposed analytical MPC tuning approach for plants can be appr...
متن کاملReliability Assessment of Shallow Domes Using a Semi-Empirical Evaluation Procedure
Like other structures, shallow domes have imperfections from the prescribed values obtained by specifications during the construction process. Specifications define some tolerance values for imperfections. Despite consideration of these values, the critical load of a dome varies for each imperfection pattern. So the reliability plays an important role in domes safety. Theoretical evaluation pro...
متن کاملA NEW APPROACH FOR EVALUATION OF SEISMIC SLOPE PERFORMANCE
The evaluation of seismic slope performance during earthquakes is important, because the failure of slope (such as an earth dam, natural slope, or constructed earth embankment) can result in significant financial losses and human. It is important, therefore, to be able to forecast such displacements induced by earthquake. However, the traditional forecasting methods, such as empirical formulae,...
متن کاملWhich Methodology is Better for Combining Linear and Nonlinear Models for Time Series Forecasting?
Both theoretical and empirical findings have suggested that combining different models can be an effective way to improve the predictive performance of each individual model. It is especially occurred when the models in the ensemble are quite different. Hybrid techniques that decompose a time series into its linear and nonlinear components are one of the most important kinds of the hybrid model...
متن کامل